SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Database scalability</title>

</head>

Database scalability

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">See also: <a href="Scalability" title="Scalability">Scalability</a></div>Database scalability is the ability of a <a href="Database" title="Database">database</a> to handle changing demands by adding/removing resources. <a href="Database" title="Database">Databases</a> use a host of techniques to cope.<a href="#cite_note-1">[1]</a> According to Marc Brooker: "a system is scalable in the range where <a href="Marginal_cost" title="Marginal cost">marginal cost</a> of additional workload is nearly constant." <a href="Serverless_computing" title="Serverless computing">Serverless</a> technologies fit this definition but you need to consider total cost of ownership not just the infra cost. <a href="#cite_note-2">[2]</a>

<div class="mw-heading mw-heading2"><h2 id="History">History</h2></div>

The initial history of database scalability was to provide service on ever smaller computers. The first database management systems such as <a href="IBM_Information_Management_System" title="IBM Information Management System">IMS</a> ran on <a href="Mainframe_computers" class="mw-redirect" title="Mainframe computers">mainframe computers</a>. The second generation, including <a href="Ingres_(database)" title="Ingres (database)">Ingres</a>, <a href="Informix" title="Informix">Informix</a>, <a href="Sybase" title="Sybase">Sybase</a>, <a href="Oracle_Rdb" title="Oracle Rdb">RDB</a> and <a href="Oracle_Database" title="Oracle Database">Oracle</a> emerged on <a href="Minicomputers" class="mw-redirect" title="Minicomputers">minicomputers</a>. The third generation, including <a href="DBase" title="DBase">dBase</a> and Oracle (again), ran on personal computers.<a href="#cite_note-:12-3">[3]</a>

During the same period, attention turned to handling more data and more demanding workloads. One key software innovation in the late 1980s was to reduce update locking granularity from tables and disk blocks to individual rows. This eliminated a critical scalability bottleneck, as coarser locks could delay access to rows even though they were not directly involved in a transaction. Earlier systems were completely insensitive to increasing resources.<a href="#cite_note-:22-4">[4]</a>

Once software limitations had been addressed, attention turned to hardware. Innovation occurred in many areas. The first was to support <a href="Multiprocessor" class="mw-redirect" title="Multiprocessor">multiprocessor computers</a>. This involved allowing multiple processors to handle database requests simultaneously, without blocking each other. This evolved into support for <a href="Multi-core_processor" title="Multi-core processor">multi-core processors</a>.

A much more significant change involved allowing <a href="Distributed_transaction" title="Distributed transaction">distributed transactions</a> to affect data stored on separate computers, using the <a href="Two-phase_commit" class="mw-redirect" title="Two-phase commit">two-phase commit</a> protocol, establishing the <a href="Shared-nothing_architecture" title="Shared-nothing architecture">shared-nothing architecture</a>.<a href="#cite_note-5">[5]</a>

Still later, Oracle introduced the shared-everything architecture, which provided full functionality on multi-server clusters.<a href="#cite_note-6">[6]</a>

Another innovation was storing copies of tables on multiple computers (<a href="Replication_(computing)" title="Replication (computing)">database replication</a>), which both improved availability (processing could continue on a copy even if the main system was unavailable) and scalability particularly for query/analysis, in that requests could be routed to the copy if the primary reached its capacity.<a href="#cite_note-:32-7">[7]</a>

In the early twenty-first century, <a href="NoSQL" title="NoSQL">NoSQL</a> systems gained favor over relational databases for some workloads. Motivations included still greater scalability and support for documents and other "non-relational" data types. Often sacrificed was the strict ACID consistency protocols that guaranteed perfect consistency at all times in favor of <a href="Eventual_consistency" title="Eventual consistency">eventual consistency</a> that ensured that all nodes would eventually return the latest data. Some even allowed for transactions to occasionally be lost, as long as the system could handle sufficiently many requests.<a href="#cite_note-8">[8]</a> The most prominent early system was Google's <a href="Bigtable" title="Bigtable">BigTable</a>/<a href="MapReduce" title="MapReduce">MapReduce</a>, developed in 2004. It achieved near-linear scalability across multiple <a href="Server_farm" title="Server farm">server farms</a>, at the cost of features such as multi-row transactions and joins.<a href="#cite_note-9">[9]</a>

In 2007, the first <a href="NewSQL" title="NewSQL">NewSQL</a> system, <a href="H-Store" title="H-Store">H-Store</a>, was developed. NewSQL systems attempt to combine NoSQL scalability with ACID transactions and SQL interfaces.<a href="#cite_note-aslett20122-10">[10]</a>

<div class="mw-heading mw-heading2"><h2 id="Dimensions">Dimensions</h2></div>

Database <a href="Scalability" title="Scalability">scalability</a> has three basic dimensions: amount of data, volume of requests and size of requests. Requests come in many sizes: transactions generally affect small amounts of data, but may approach thousands per second; analytic queries are generally fewer, but may access more data. A related concept is elasticity, the ability of a system to transparently add and subtract capacity to meet changing workloads.<a href="#cite_note-:02-11">[11]</a>

<div class="mw-heading mw-heading3"><h3 id="Vertical">Vertical</h3></div>

Vertical database scaling implies that the database system can fully exploit maximally configured systems, including typically multiprocessors with large memories and vast storage capacity. Such systems are relatively simple to administer, but may offer reduced availability. However, any single computer has a maximum configuration. If workloads expand beyond that limit, the choices are either to migrate to a different, still larger system, or to rearchitect the system to achieve horizontal scalability.<a href="#cite_note-:02-11">[11]</a>

<div class="mw-heading mw-heading3"><h3 id="Horizontal">Horizontal</h3></div>

Horizontal database scaling involves adding more servers to work on a single workload. Most horizontally scalable systems come with functionality compromises. If an application requires more functionality, migration to a vertically scaled system may be preferable.<a href="#cite_note-:02-11">[11]</a>

<div class="mw-heading mw-heading2"><h2 id="Techniques">Techniques</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Hardware">Hardware</h3></div>

Databases run on individual hardware ranging in capacity from smartwatches to supercomputers to multiple transparently reconfigurable server farms.<a href="#cite_note-:12-3">[3]</a> Databases also scaled vertically to run on 64-bit <a href="Microprocessor" title="Microprocessor">microprocessors</a>, <a href="Multi-core_(computing)" class="mw-redirect" title="Multi-core (computing)">multi-core</a> CPUs, and large <a href="Symmetric_multiprocessing" title="Symmetric multiprocessing">SMP multiprocessors</a>, using <a href="Thread_(computer_science)" class="mw-redirect" title="Thread (computer science)">multi-threaded</a> implementations.

<div class="mw-heading mw-heading3"><h3 id="Contention">Contention</h3></div>

Fully exploiting a hardware configuration requires a variety of locking techniques, ranging from locking an entire database to entire tables to disk blocks to individual table rows. The appropriate lock granularity depends on the workload. The smaller the object that is locked, the less the chance of database requests blocking each other, while the hardware idles. Typically row locks are necessary to support high volume transaction processing applications at the cost of processing overhead to manage the larger number of locks.<a href="#cite_note-:22-4">[4]</a>

Further, some systems ensure that a query sees a time-consistent view of the database by locking data that a query is examining to prevent an update from modifying it, stalling work. Alternatively, some databases use <a href="Multiversion_concurrency_control" title="Multiversion concurrency control">multi-version read consistency</a> to avoid (blocking) read locks while still providing consistent query results.<a href="#cite_note-12">[12]</a>

Another potential bottleneck can occur in some systems when many requests attempt to access the same data at the same time. For example, in OLTP systems, many transactions may attempt to insert data into the same table at the same time. In a shared nothing system, at any given moment, all such inserts are processed by the single server that manages that partition (shard) of the table, possibly overwhelming it, while the rest of the system has little to do. Many such tables use a sequence number as their primary key that increases for each new inserted row. The index for that key can also experience contention (overheat) as it processes those inserts. One solution for this is to <a href="Reverse_index" title="Reverse index">reverse the digits of the primary key</a>. This spreads the inserts into both the table and the key across multiple parts of the database.<a href="#cite_note-13">[13]</a>

<div class="mw-heading mw-heading3"><h3 id="Partitioning">Partitioning</h3></div>

A basic technique is to <a href="Partition_(database)" title="Partition (database)">split</a> large tables into multiple partitions based on ranges of values in a key field. For example, the data for each year could be held on a separate disk drive or on a separate computer. Partitioning removes limits on the sizes of a single table.

<div class="mw-heading mw-heading3"><h3 id="Replication">Replication</h3></div>

Replicated databases maintain copies of tables or databases on multiple computers. This scaling technique is particularly convenient for seldom or never-updated data, such as transaction history or tax tables.<a href="#cite_note-:32-7">[7]</a>

<div class="mw-heading mw-heading3"><h3 id="Clustered_computers">Clustered computers</h3></div>

A variety of approaches are used to scale beyond the limits of a single computer. <a href="Hewlett_Packard_Enterprise" title="Hewlett Packard Enterprise">HP Enterprise</a>'s <a href="NonStop_SQL" title="NonStop SQL">NonStop SQL</a> uses the shared nothing architecture in which neither data nor memory are shared across server boundaries. A coordinator routes database requests to the correct server. This architecture provides near-linear scalability.

The widely supported <a href="X/Open_XA" title="X/Open XA">X/Open XA</a> standard employs a global transaction monitor to coordinate <a href="Distributed_transaction" title="Distributed transaction">distributed transactions</a> among semi-autonomous XA-compliant transaction resources.

<a href="Oracle_RAC" title="Oracle RAC">Oracle RAC</a> uses a different model to achieve scalability, based on a "shared-everything" architecture. This approach incorporates the <a href="Shared_disk_architecture" class="mw-redirect" title="Shared disk architecture">shared disk</a> approach that allows multiple computers to access any disk in the cluster. <a href="Network-attached_storage" title="Network-attached storage">Network-attached storage (NAS)</a> and <a href="Storage_area_network" title="Storage area network">Storage area networks (SANs)</a> coupled with local area networks and <a href="Fibre_Channel" title="Fibre Channel">Fibre Channel</a> technology enable such configurations. The approach includes a "shared" logical cache in which data that has been cached in memory on server is made available to other servers without requiring them to again read the data from disk. Each page is moved from server to server to satisfy requests. Updates generally happen very quickly so that a "popular" page can be updated by multiple transactions with little delay. This approach is claimed to support clusters containing up to 100 servers.<a href="#cite_note-14">[14]</a>

Some researchers question the inherent limitations of <a href="Relational_database_management_system" class="mw-redirect" title="Relational database management system">relational database management systems</a>. <a href="GigaSpaces" title="GigaSpaces">GigaSpaces</a>, for example, contends that <a href="Space-based_architecture" title="Space-based architecture">space-based architecture</a> is required to achieve performance and scalability. <a href="Base_One" class="mw-redirect" title="Base One">Base One</a> makes the case for extreme scalability within mainstream relational database technology.<a href="#cite_note-15">[15]</a>

<ul><li><a href="Relational_database" title="Relational database">Relational database</a></li>

<li><a href="Scalability" title="Scalability">Scalability</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFBondi2000" class="citation conference cs1">Bondi, André B. (2000). Characteristics of scalability and their impact on performance. Proceedings of the second international workshop on Software and performance – WOSP '00. p. 195. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F350391.350432">10.1145/350391.350432</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>158113195X</bdi>.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite class="citation book cs1">Building Serverless Applications on Knative. O'Reilly Media. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781098142049</bdi>.</cite>

</li>

<li id="cite_note-:12-3">^ <a href="#cite_ref-:12_3-0">a</a> <a href="#cite_ref-:12_3-1">b</a> <cite id="CITEREFChopra2010" class="citation book cs1">Chopra, Rajiv (2010). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=FTUJNA4lLdAC&pg=PA33">Database Management System (DBMS)A Practical Approach</a>. S. Chand Publishing. p. 33. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9788121932455</bdi>.</cite>

</li>

<li id="cite_note-:22-4">^ <a href="#cite_ref-:22_4-0">a</a> <a href="#cite_ref-:22_4-1">b</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.dba-oracle.com/t_row_locks_vs_table_locks.htm">"Row locks vs table locks in Oracle"</a>. www.dba-oracle.com. Retrieved 2019-04-11.</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://web.archive.org/web/20150424040705/http://www.solidfire.com/blog/the-advantages-of-a-shared-nothing-architecture-for-truly-non-disruptive-upgrades/">"The Advantages of a Shared Nothing Architecture for Truly Non-Disruptive Upgrades"</a>. solidfire.com. 2014-09-17. Archived from <a rel="nofollow" class="external text" href="http://www.solidfire.com/blog/the-advantages-of-a-shared-nothing-architecture-for-truly-non-disruptive-upgrades/">the original</a> on 2015-04-24. Retrieved 2015-04-21.</cite>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://docs.oracle.com/cd/B28359_01/rac.111/b28254/admcon.htm#RACAD7150">"Real Application Clusters Administration and Deployment Guide"</a>. docs.oracle.com. Retrieved 2019-04-11.</cite>

</li>

<li id="cite_note-:32-7">^ <a href="#cite_ref-:32_7-0">a</a> <a href="#cite_ref-:32_7-1">b</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://www.brianstorti.com/replication/">"A Primer on Database Replication"</a>. www.brianstorti.com. Retrieved 2019-04-11.</cite>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite id="CITEREFMartin_Zapletal2015" class="citation journal cs1">Martin Zapletal (2015-06-11). <a rel="nofollow" class="external text" href="https://www.slideshare.net/MartinZapletal/zapletal-martinlargevolumedataanalytics">"Large volume data analysis on the Typesafe Reactive Platform"</a>.</cite> <code class="cs1-code">{{cite journal}}</code>: Cite journal requires <code class="cs1-code">|journal=</code> (help)

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://cloud.google.com/bigtable/docs/overview">"Overview of Cloud Bigtable | Cloud Bigtable Documentation"</a>. Google Cloud. Retrieved 2019-04-11.</cite>

</li>

<li id="cite_note-aslett20122-10"><a href="#cite_ref-aslett20122_10-0">^</a> <cite id="CITEREFAslett2011" class="citation web cs1">Aslett, Matthew (2011). <a rel="nofollow" class="external text" href="http://cs.brown.edu/courses/cs227/archives/2012/papers/newsql/aslett-newsql.pdf">"How Will The Database Incumbents Respond To NoSQL And NewSQL?"</a> (PDF). 451 Group (published 2011-04-04). Retrieved 2012-07-06.</cite>

</li>

<li id="cite_note-:02-11">^ <a href="#cite_ref-:02_11-0">a</a> <a href="#cite_ref-:02_11-1">b</a> <a href="#cite_ref-:02_11-2">c</a> <cite id="CITEREFBranson2016" class="citation web cs1">Branson, Tony (2016-12-06). <a rel="nofollow" class="external text" href="https://www.infosecurity-magazine.com:443/opinions/a-glimpse-into-database-scalability/">"Two main approaches to database scalability"</a>. Infosecurity Magazine. Retrieved 2019-04-11.</cite>

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://clojure.org/reference/refs">"Clojure - Refs and Transactions"</a>. clojure.org. Retrieved 2019-04-12.</cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://richardfoote.wordpress.com/2008/01/14/introduction-to-reverse-key-indexes-part-i/">"Introduction To Reverse Key Indexes: Part I"</a>. Richard Foote's Oracle Blog. 2008-01-14. Retrieved 2019-04-13.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.oracle.com/technology/products/database/clustering/pdf/ds_rac11g.pdf">"clustering"</a> (PDF). Oracle.com. Retrieved 2012-11-07.</cite>

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> <cite id="CITEREFBase_One2007" class="citation web cs1">Base One (2007). <a rel="nofollow" class="external text" href="http://www.boic.com/scalability.htm">"Database Scalability - Dispelling myths about the limits of database-centric architecture"</a>. Retrieved May 23, 2007.</cite>

</li>

</ol></div>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

</div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2024-10-04" href="https://en.wikipedia.org/wiki/?title=Database_scalability&oldid=1249313843">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>